Virtualization
There are two main categories of virtualization: 1) Computing Machine Virtualization 2) Resource Virtualization.
Computing machine virtualization involves definition and virtualization of multiple operating system (OS) instances and application stacks into partitions within a host system.
Resource virtualization refers to the abstraction of computer peripheral functions. There are two main types of Resource virtualization: 1) Storage Virtualization 2) System Memory-Mapped I/O Virtualization.
Storage virtualization involves the abstraction and aggregation of multiple physical storage components into logical storage pools that can then be allocated as needed to computing machines.
System Memory-Mapped I/O virtualization involves the abstraction of a wide variety of I/O resources, including but not limited to bridge devices, memory controllers, display controllers, input devices, multi-media devices, serial data acquisition devices, video devices, audio devices, modems, etc. that are assigned a location in host processor memory. Examples of System Memory-Mapped I/O Virtualization are exemplified by PCI Express I/O Virtualization (IOV) and applicant's technology referred to as i-PCI.
PCIe and PCIe I/O Virtualization
PCI Express (PCIe), as the successor to PCI bus, has moved to the forefront as the predominant local host bus for computer system motherboard architectures. A cabled version of PCI Express allows for high performance directly attached bus expansion via docks or expansion chassis. These docks and expansion chassis may be populated with any of the myriad of widely available PCI Express or PCI/PCI-X bus adapter cards. The adapter cards may be storage oriented (i.e. Fibre Channel, SCSI), video processing, audio processing, or any number of application specific Input/Output (I/O) functions. A limitation of PCI Express is that it is limited to direct attach expansion.
The PCI Special Interest Group (PCI-SIG) has defined single root and multi-root I/O virtualization sharing specifications.
The single-root specification defines the means by which a host, executing multiple systems instances may share PCI resources. In the case of single-root IOV, the resources are typically but not necessarily accessed via expansion slots located on the system motherboard itself and housed in the same enclosure as the host.
The multi-root specification on the other hand defines the means by which multiple hosts, executing multiple systems instances on disparate processing components, may utilize a common PCI Express (PCIe) switch in a topology to connect to and share common PCI Express resources. In the case of PCI Express multi-root IOV, resources are accessed and shared amongst two or more hosts via a PCI Express fabric. The resources are typically housed in a physically separate enclosure or card cage. Connections to the enclosure are via a high-performance short-distance cable as defined by the PCI Express External Cabling specification. The PCI Express resources may be serially or simultaneously shared.
A key constraint for PCIe I/O virtualization is the severe distance limitation of the external cabling. There is no provision for the utilization of networks for virtualization.
i-PCI
This invention builds and expands on applicant's technologydiscosed as “i-PCI” in commonly assigned copending U.S. patent application Ser. No. 12/148,712, the teachings of which are incorporated herein by reference. This patent application presents i-PCI as a new technology for extending computer systems over a network. The i-PCI protocol is a hardware, software, and firmware architecture that collectively enables virtualization of host memory-mapped I/O systems. For a PCI-based host, this involves extending the PCI I/O system architecture based on PCI Express.
The i-PCI protocol extends the PCI I/O System via encapsulation of PCI Express packets within network routing and transport layers and Ethernet packets and then utilizes the network as a transport. The network is made transparent to the host and thus the remote I/O appears to the host system as an integral part of the local PCI system architecture. The result is a virtualization of the host PCI System. The i-PCI protocol allows certain hardware devices (in particular I/O devices) native to the host architecture (including bridges, I/O controllers, and I/O cards) to be located remotely. FIG. 1 shows a detailed functional block diagram of a typical host system connected to multiple remote I/O chassis. An i-PCI host bus adapter card [101] installed in a host PCI Express slot [102] interfaces the host to the network. An i-PCI remote bus adapter card [103] interfaces the remote PCI Express bus resources to the network.
There are three basic implementations of i-PCI:
1. i-PCI: This is the TCP/IP implementation, utilizing IP addressing and routers. This implementation is the least efficient and results in the lowest data throughput of the three options, but it maximizes flexibility in quantity and distribution of the I/O units. Refer to FIG. 2 for an i-PCI IP-based network implementation block diagram.
2. i(e)-PCI: This is the LAN implementation, utilizing MAC addresses and Ethernet switches. This implementation is more efficient than the i-PCI TCP/IP implementation, but is less efficient than i(dc)-PCI. It allows for a large number of locally connected I/O units. Refer to FIG. 3 for an i(e)-PCI MAC-Address switched LAN implementation block diagram.
3. i(dc)-PCI. Referring to FIG. 4, this is a direct physical connect implementation, utilizing Ethernet CAT-x cables. This implementation is the most efficient and highest data throughput option, but it is limited to a single remote I/O unit. The standard implementation currently utilizes 10 Gbps Ethernet (802.3an) for the link [401], however, there are two other lower performance variations. These are designated the “Low End” LE(dc) or low performance variations, typically suitable for embedded or cost sensitive installations:
The first low end variation is LE(dc) Triple link Aggregation 1 Gbps Ethernet (802.3ab) [402] for mapping to single-lane 2.5 Gbps PCI Express [403] at the remote I/O.
A second variation is LE(dc) Single link 1 Gbps Ethernet [404] for mapping single-lane 2.5 Gbps PCI Express [405] on a host to a legacy 32-bit/33 MHz PCI bus-based [406] remote I/O.
A wireless version is also an implementation option for i-PCI. In a physical realization, this amounts to a wireless version of the Host Bus Adapter (HBA) and Remote Bus Adapter (RBA).
The i-PCI protocol describes packet formation via encapsulation of PCI Express Transaction Layer packets (TLP). The encapsulation is different depending on which of the implementations is in use. If IP is used as a transport (as illustrated in FIG. 2), the end encapsulation is within TCP, IP, and Ethernet headers and footers. If a switched LAN is used as a transport, the end encapsulation is within Ethernet data link and physical layer headers and footers. If a direct connect is implemented, the end encapsulation is within the Ethernet physical layer header and footer. FIG. 5 shows the high-level overall concept of the encapsulation technique, where TCP/IP is used as a transport.
This invention builds and expands on teachings introduced as “i-PCI” in commonly assigned copending U.S. patent application Ser. Nos. 12/148,712 and 12/286,796 the teachings of which are incorporated herein by reference.